An Equivalence between Sigmoidal Gain Scaling and Training with Noisy (Jittered) Input Data

Training with additive input noise (jitter) is a commonly used heuristic for improving generalization in layered perceptron artifiaal neural networks. One result of training with jitter is that the effective target function is the convolution of the actual target and the noise density. For many noise densities, this is approximately equivalent to a smoothing regularization. A drawback of training with jitter, in comparison with the unjittered cme, is that many more sample presentations are required in order to average over the noise and estimate the expected response. In this paper, we demonstrate that the expected effect of jitter can be computed, in certain cases, by a simple scaling of the sigmoid nonlinearities. Application of this technique to a singlehidden-layer perceptron with a linear output is considered.